Skip to content

fix: keep Python wrappers valid when ObjCryst++ releases sub-objects - #133

Merged
sbillinge merged 4 commits into
mainfrom
vincefn-objcryst-release-hooks
Nov 16, 2026
Merged

sbillinge merged 4 commits into
mainfrom
vincefn-objcryst-release-hooks

Conversation

@vincefn

@vincefn vincefn commented Nov 16, 2026

Copy link
Copy Markdown
Collaborator

What problem does this PR address?

Fixes #121 and #132: ObjCryst++ internally deletes sub-objects (reflection profiles, refinable parameters, scatterers, scattering powers, molecule atoms/restraints) that Python wrappers still reference. The result was use-after-free, double free, or nanobind returning a stale wrapper of the wrong type for a new object allocated at the same address.

This relies on the ReleaseObj() hook API added in vincefn/objcryst#106 (merged; extern/objcryst updated to 0f8dfef).

  • New src/extensions/nb_release_hooks.cpp installs the ObjCryst++ release hooks:
    • if Python holds a wrapper of the released object, the wrapper takes ownership (the object is deleted when the wrapper dies); otherwise ObjCryst++ deletes it as before;
    • wrappers of released RefinablePar are invalidated (raise TypeError), since their value points into the former owner's storage;
    • kept atoms pin their scattering power; kept restraints/rigid groups pin their atoms.
  • Crystal.RemoveScatterer, Crystal.RemoveScatteringPower (when unused) and Molecule.Remove* now delete the removed objects instead of leaking them.
  • Removed the GetPar SetDeleteRefParInDestructor(0) workaround.

Validation:

  • pytest tests — 214 passed (also passes under MallocScribble=1 MallocGuardEdges=1 and with an AddressSanitizer build).
  • Added 10 regression tests (SetProfile/SetReflectionProfilePar, Background SetInterpPoints, TextureMarchDollase.DeletePhase, ConnectAtoms, Crystal() destructor, RemoveScatterer, XMLInput, Molecule.RemoveAtom); 9 of them crash or fail on main.

Known remaining (pre-existing, not addressed here): ScatteringPowerRegistry.GetObj returns a wrapper at the unadjusted virtual-base address.

What should the reviewer(s) do?

Review the hook design and the Remove* behaviour change. Do not auto-merge.

  • This PR introduces a public-facing change (e.g., figures, CLI input/output, API).
    • Documentation (e.g., tutorials, examples, README) has been updated.
    • A tracking issue or plan to update documentation exists.

vincefn and others added 4 commits November 16, 2026 13:35
Install ObjCryst++ ReleaseObj() hooks (vincefn/objcryst#106) so that
sub-objects deleted internally by ObjCryst++ (reflection profiles,
scatterers, scattering powers, refinable parameters, molecule atoms,
restraints and rigid groups) no longer leave dangling Python wrappers.

- If a Python wrapper exists, the hook hands ownership to it instead of
  letting ObjCryst++ delete the object; otherwise ObjCryst++ deletes it.
- Wrappers of released RefinablePar objects are invalidated, since their
  value pointer refers to storage of the former owner.
- Atoms pin their scattering power, and released restraints and rigid
  groups pin their atoms while the molecule is alive.
- Remove the GetPar SetDeleteRefParInDestructor(0) workaround.
- Crystal.RemoveScatterer, RemoveScatteringPower (when unused) and the
  Molecule.Remove* methods now delete unreferenced objects instead of
  leaking them.
- Update extern/objcryst to a3d75af (vincefn-object-release-hook).

Fixes #121, #132.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@codecov

codecov Bot commented Nov 16, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.87%. Comparing base (0a305f7) to head (0eed08c).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #133      +/-   ##
==========================================
+ Coverage   98.80%   98.87%   +0.07%     
==========================================
  Files          22       22              
  Lines        2343     2495     +152     
==========================================
+ Hits         2315     2467     +152     
  Misses         28       28              
Files with missing lines Coverage Δ
tests/test_crystal.py 99.37% <100.00%> (+0.38%) ⬆️
tests/test_molecule.py 99.82% <100.00%> (+0.01%) ⬆️
tests/test_powderpattern.py 99.71% <100.00%> (+0.02%) ⬆️
tests/test_reflectionprofile.py 99.57% <100.00%> (+0.04%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@vincefn

vincefn commented Nov 16, 2026

Copy link
Copy Markdown
Collaborator Author

@sbillinge @SYMSCAE I let you check this - the hooks introduced are a bit long, but this should cover most cases.

@sbillinge sbillinge left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one small inline quesstion @vincefn

{
pars[0] = &m.GetPar(&atom.X());
pars[1] = &m.GetPar(&atom.Y());
pars[2] = &m.GetPar(&atom.Z());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are there any other atom attributes that have be treated the same way, such as ADPs and occupancies?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In a Molecule we need this for the xyz atom coordinates because they are used as refinable parameters, so there are references pointing to them in the Molecule (not in a MolAtom because that class is not a RefinableObj). That's not the case for other parameters.
For ADPs these belong inside a ScatteringPower, and handled properly there.

Of course this is still a hook, the correct longer-term fix would be to implement reference counting in objcryst.

@sbillinge

Copy link
Copy Markdown
Contributor

@SYMSCAE please could you look at this. Also, do we have any of the same issues with libobjcryst or pdfit2 bindings?

@sbillinge

Copy link
Copy Markdown
Contributor

@SYMSCAE it would be great if you could look at this. However, I will merge it and push out an rc release and see what happens in the matrix.....

@sbillinge
sbillinge merged commit 88fd4f4 into main Nov 16, 2026
7 checks passed
@sbillinge

Copy link
Copy Markdown
Contributor

OK, I pushed out 2026.2.0-rc.1

@SYMSCAE

SYMSCAE commented Nov 16, 2026

Copy link
Copy Markdown
Contributor

LGTM.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Ready for 2026.2.0 PyPI/GitHub release

3 participants